AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	0.554373	1.74
	2_DecisionTree	Decision Tree	logloss	0.284821	19
the best	3_Default_Xgboost	Xgboost	logloss	0.00762922	62.8
	4_Default_NeuralNetwork	Neural Network	logloss	0.169421	10.7
	5_Default_RandomForest	Random Forest	logloss	0.180638	21.14
	Ensemble	Ensemble	logloss	0.00762922	1.56

AutoML Performance

AutoML Performance Boxplot

Features Importance

features importance across models

Spearman Correlation of Models

models spearman correlation

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

n_jobs: -1
dense_1_size: 32
dense_2_size: 16
learning_rate: 0.05
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

9.4 seconds

Metric details

	score	threshold
logloss	0.169421	nan
auc	0.977523	nan
f1	0.8791	0.441499
accuracy	0.941655	0.441499
precision	1	0.999897
recall	1	2.04069e-11
mcc	0.840684	0.441499

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.169421	nan
auc	0.977523	nan
f1	0.8791	0.441499
accuracy	0.941655	0.441499
precision	0.884906	0.441499
recall	0.873371	0.441499
mcc	0.840684	0.441499

Confusion matrix (at threshold=0.441499)

	Predicted as Medical	Predicted as Surgical
Labeled as Medical	3226	122
Labeled as Surgical	136	938

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

n_jobs: -1
criterion: gini
max_features: 0.9
min_samples_split: 30
max_depth: 4
eval_metric_name: logloss
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

19.6 seconds

Metric details

	score	threshold
logloss	0.180638	nan
auc	0.978632	nan
f1	0.859004	0.518382
accuracy	0.931479	0.518382
precision	1	0.962334
recall	1	0.0116859
mcc	0.813747	0.518382

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.180638	nan
auc	0.978632	nan
f1	0.859004	0.518382
accuracy	0.931479	0.518382
precision	0.858605	0.518382
recall	0.859404	0.518382
mcc	0.813747	0.518382

Confusion matrix (at threshold=0.518382)

	Predicted as Medical	Predicted as Surgical
Labeled as Medical	3196	152
Labeled as Surgical	151	923

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Dependence from Fold 1

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

SHAP worst decisions class 0 from Fold 1

Top-10 Best decisions for class 0 (Fold 1)

SHAP best decisions class 0 from Fold 1

Top-10 Worst decisions for class 1 (Fold 1)

SHAP worst decisions class 1 from Fold 1

Top-10 Best decisions for class 1 (Fold 1)

SHAP best decisions class 1 from Fold 1

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

n_jobs: -1
criterion: gini
max_depth: 3
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

17.5 seconds

Metric details

	score	threshold
logloss	0.284821	nan
auc	0.903691	nan
f1	0.745037	0.251308
accuracy	0.870647	0.694369
precision	1	0.928019
recall	1	0
mcc	0.662072	0.251308

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.284821	nan
auc	0.903691	nan
f1	0.653333	0.694369
accuracy	0.870647	0.694369
precision	0.935764	0.694369
recall	0.501862	0.694369
mcc	0.625307	0.694369

Confusion matrix (at threshold=0.694369)

	Predicted as Medical	Predicted as Surgical
Labeled as Medical	3311	37
Labeled as Surgical	535	539

Learning curves

Decision Tree

Tree #1

Rules

if (Type of Admission > 0.5) and (Total Costs <= 11941.71) and (CCS Procedure Description > 24.5) then class: Medical (proba: 95.81%) | based on 6,707 samples

if (Type of Admission > 0.5) and (Total Costs > 11941.71) and (CCS Procedure Code <= 176.5) then class: Medical (proba: 53.93%) | based on 2,253 samples

if (Type of Admission <= 0.5) and (APR DRG Code <= 552.5) and (CCS Procedure Code > 0.5) then class: Surgical (proba: 92.8%) | based on 1,681 samples

if (Type of Admission > 0.5) and (Total Costs > 11941.71) and (CCS Procedure Code > 176.5) then class: Medical (proba: 96.57%) | based on 1,137 samples

if (Type of Admission <= 0.5) and (APR DRG Code > 552.5) and (APR DRG Code <= 906.5) then class: Medical (proba: 97.23%) | based on 723 samples

if (Type of Admission > 0.5) and (Total Costs <= 11941.71) and (CCS Procedure Description <= 24.5) then class: Medical (proba: 60.03%) | based on 673 samples

if (Type of Admission <= 0.5) and (APR DRG Code <= 552.5) and (CCS Procedure Code <= 0.5) then class: Medical (proba: 100.0%) | based on 75 samples

if (Type of Admission <= 0.5) and (APR DRG Code > 552.5) and (APR DRG Code > 906.5) then class: Surgical (proba: 100.0%) | based on 14 samples

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Dependence from Fold 1

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

SHAP worst decisions class 0 from Fold 1

Top-10 Best decisions for class 0 (Fold 1)

SHAP best decisions class 0 from Fold 1

Top-10 Worst decisions for class 1 (Fold 1)

SHAP worst decisions class 1 from Fold 1

Top-10 Best decisions for class 1 (Fold 1)

SHAP best decisions class 1 from Fold 1

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

n_jobs: -1
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

0.7 seconds

Metric details

	score	threshold
logloss	0.554373	nan
auc	0.5	nan
f1	0.39083	0.21857
accuracy	0.242877	0.21857
precision	0.242877	0.21857
recall	1	0.21857
mcc	0	0.21857

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.554373	nan
auc	0.5	nan
f1	0.39083	0.21857
accuracy	0.242877	0.21857
precision	0.242877	0.21857
recall	1	0.21857
mcc	0	0.21857

Confusion matrix (at threshold=0.21857)

	Predicted as Medical	Predicted as Surgical
Labeled as Medical	0	3348
Labeled as Surgical	0	1074

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	3

Metric details

	score	threshold
logloss	0.00762922	nan
auc	0.99991	nan
f1	0.995812	0.235033
accuracy	0.997965	0.235033
precision	1	0.998556
recall	1	1.39117e-07
mcc	0.994468	0.235033

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.00762922	nan
auc	0.99991	nan
f1	0.995812	0.235033
accuracy	0.997965	0.235033
precision	0.995349	0.235033
recall	0.996276	0.235033
mcc	0.994468	0.235033

Confusion matrix (at threshold=0.235033)

	Predicted as Medical	Predicted as Surgical
Labeled as Medical	3343	5
Labeled as Surgical	4	1070

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

n_jobs: -1
objective: binary:logistic
eta: 0.075
max_depth: 6
min_child_weight: 1
subsample: 1.0
colsample_bytree: 1.0
eval_metric: logloss
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

61.2 seconds

Metric details

	score	threshold
logloss	0.00762922	nan
auc	0.99991	nan
f1	0.995812	0.235033
accuracy	0.997965	0.235033
precision	1	0.998556
recall	1	1.39117e-07
mcc	0.994468	0.235033

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.00762922	nan
auc	0.99991	nan
f1	0.995812	0.235033
accuracy	0.997965	0.235033
precision	0.995349	0.235033
recall	0.996276	0.235033
mcc	0.994468	0.235033

Confusion matrix (at threshold=0.235033)

	Predicted as Medical	Predicted as Surgical
Labeled as Medical	3343	5
Labeled as Surgical	4	1070